An Approach to Assembling Optimal Multistage Testing Modules on the Fly

This study presents a new adaptive multistage testing method that replaces the preassembled test module with a test module assembled on the fly after each stage. In this method, a test module for each stage is shaped to come as close as possible to the normal density function of the interim proficiency estimate and its standard error. We will hereafter call the new method ‘multistage test by shaping’ (MST-S) and refer to the traditional multistage test as MST by routing (MST-R). The new MST-S offers the advantages of both MST-R and computerized adaptive testing (CAT). With MST-S, the difficulty of a test module always centers on the latest interim proficiency estimate, which means it potentially can administer a test module that is more efficiently adapted to the individual compared with MST-R. Because test items are not necessarily limited to a certain stage but instead are available for use in any stage, the number of items required to implement MST-S can be much smaller than those required in MST-R. If desired, MST-S also can allow examinees to move back and forth within each module, as they can do now in MST-R. This study consists of a series of simulation studies that were conducted to evaluate the performance of MST-S in comparison with MST-R and CAT.